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Abstract 

On the basis of hardware conditions, the 3D digitization of 
cultural heritage entities still requires a lot of human re- 
sources. In a short period of time, it is difficult to obtain a 
large number of 3D models of cultural heritage with conser- 
vation and communication value under the current main- 
stream production process. The ability of smartphones in 
photography and video has been enhanced by the mutual 
emphasis of manufacturers and consumers, and it is gradu- 
ally becoming possible to generate reliable 3D models from 
the photos taken by them. In addition, the images generated 
by the large number of visitors taking pictures of objects and 
other content in museums provide a large amount of poten- 
tial data for the digitization of cultural heritage. This work 
aims to examine the reliability of 3D entities of cultural her- 
itage generated using images of cultural objects recorded by 
visitors in museums through mobile camera devices such 
as smartphones and to verify the value of the application of 
the models. The method can provide a way to quickly record 
large amounts of data for cultural heritage conservation with 
sufficient accuracy and ease of use, making it a potential al- 
ternative to traditional methods that rely on specialized per- 
sonnel and specialized equipment, such as photogrammetry 
or laser scanning with specialized equipment. 
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Introduction 


Background 

In the digital era, the primary work of systematically con- 
structing digital resources of cultural heritage is to carry 
out the digitization project of cultural relics and realize the 
transformation of cultural relics from material form to digital 
form. Three-dimensional digitization has become a common 
practice in cultural heritage conservation, but there are still 
some problems. On the one hand, the equipment used for 
digitization of cultural relics, such as laser scanners or cam- 
era arrays, is mostly expensive and requires specification of 
the operation, which requires a certain amount of money 


and human resources. On the other hand, because the work- 
load of digitizing cultural relics is large and difficult, the staff 
needs to have basic operational knowledge and experience in 
data acquisition and processing. Therefore, the work is usual- 
ly done by cultural and museum institutions or professional 
teams. Cultural and museum institutions usually use profes- 
sional recording equipment and follow standardized operat- 
ing procedures to complete the digitization of cultural relics. 
Both the data kept within the institution and the models open 
for public viewing have high-quality information recorded in 
terms of form and color. 

Today, the improvement of digitization technology has 
made digitization significantly less difficult. Structured light 
(Rocchini, 2001), encoded light methods, and photogramme- 
try are all mainstream methods for 3D scanning. Among them, 
photogrammetry, with the optimization of algorithms and the 
improvement of the performance of image acquisition equip- 
ment, has had the effect of significantly reducing the cost of 
both acquisition and computing (Pavlidis, 2007). Individual use 
of camera equipment, with software such as RealityCapture 
or Metashape and other related processing software, allows 
you to do the three-cdimensional digitization of real entities. 
Even ordinary people can go digital by using their smartphones 
(Boboc, 2019). This undoubtedly reduces the resources and 
learning costs of the process of digitizing the real world in 3D, 
making everyone a mover of 3D information in the physical 
world and a producer of assets in the digital world. Among the 
models in the public 3D content center sketchfab, the model 
assets under the “Cultural Heritage and History” tab are both 
produced by professional cultural and historical institutions 
and also by users themselves. The majority of them are pro- 
duced using multi-image generation methods. 

Even in the three-dimensional technology is so convenient 
now, there are still a large number of cultural relics in cultural 
and museum institutions have not been established a digital 
model file, but also can not get a large number of preservation, 
dissemination value of cultural heritage digital model in a short 
period of time. This is both because the calculation and pro- 
cessing of data requires considerable computing power and 
time, and at the same time requires considerable human re- 
sources to complete the image acquisition work used to gen- 
erate the model. At the same time, as mobile camera tools are 
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widely used by museum visitors, more and more “wild digital 
images” of cultural heritage are being produced (Makantasis, 
2016). Widely uploaded to online platforms such as social me- 
dia or stored on users’ devices, they are not considered valid 
data for the preservation of cultural heritage, but constitute 
potential content for its digitization. With the help of device 
updates, visitors have generally become capable of digitizing 
cultural heritage at the two-dimensional level. Although they 
are still not widely practiced at the three-dimensional level, an 
opportunity does exist to enable every visitor to the museum 
to participate in the preservation of cultural heritage. This en- 
riches the museum experience while saving institutions sig- 
nificant human resources costs and enabling the rapid recon- 
struction of digital assets (Mingyao Ai, 2014). 


Purpose 

This research aims to check the reliability of 3D entities of 
cultural heritage generated using images of artifacts record- 
ed by visitors in museums through mobile camera devices 
such as smartphones and to verify the value of the applica- 
tion of the models. The main focus is to compare models 
generated through multiple devices (smartphones, cameras) 
and methods (structured light, encoded light, photogramme- 
try), demonstrating that phones are quite effective as data 
acquisition devices for photogrammetry for 3D reconstruc- 
tion of tangible cultural heritage, providing morphometric 
data comparable to laser scanning results and easier-to-use 
color information. 

Based on this, a participatory approach can be designed 
to help subjects or visitors complete the photography of the 
collected objects more efficiently and with high quality, and a 
real-time visual participation system is used to complete the 
guidance of the visitors’ photography and provide real-time 
feedback of the results. Finally, the quality of the obtained 
model results is checked. 


Methodology 
The study is divided into two main phases, and each phase 
contains a variety of practical and testing steps. 


Digital practices in the Hunan Museum scenario 
The first phase of the study was designed to verify the ability 
of visitors to obtain relatively accurate and usable digital as- 
sets in the current real-life museum scenario. The 15 volun- 
teers who participated in this phase need to enter the Hunan 
Provincial Museum as ordinary visitors, select artifacts dis- 
played in the museum as their digitized objects, and use their 
mobile devices (cameras or smartphones) to complete the 
multi-view photography work and finally generate a polygon 
with color information. 

Most of the participants had no experience with or knowl- 
edge of polygon modeling and had never been exposed to 3D 
inverse knowledge like photogrammetry before, so they need- 
ed to learn and practice some relevant knowledge and skills in 
advance before starting the practice, including the following: 

1. An explanation of the fundamental principles of pho- 
togrammetry to give volunteers an idea of what to ex- 
pect during the acquisition process. 

2. A photogrammetry lecture designed to help partic- 
ipants quickly master the methods and techniques 
used to generate a set of high-quality images for 3D 
modeling. Also, participants will be asked to take pho- 


tos of an object with at least three horizontal tracks 
and at least 6O photos to make sure there are enough 
images for model generation. 

3. Practical shooting exercises and model generation 
tests. Participants test their learning by taking pictures 
of pre-prepared acquisition objects and generating 
models. 


The entire learning process was kept to less than one hour, 
and most people obtained excellent-quality 3D models after 
only one practice session. Afterwards, the volunteers start- 
ed a two-hour tour and photography practice, at the end of 
which the images were collected and used to generate 3D 
models, which were then tested and evaluated. 


Figure 1. The state of aging of the collected objects in the Hunan Museum. 


Usually, the quality inspection of 3D models obtained by re- 
verse scanning is done by matching and comparing them 
with the original digital model. Since there is no absolutely 
accurate native digital model of cultural heritage entities, 
new ways and methods are needed to evaluate the latter 
models. A cross-sectional comparison of the results of mul- 
tiple models obtained by different equipment or processes 
for the same collection object to draw conclusions is the 
accuracy test method used in this study. The distance dif- 
ference of the meshes can be used as a basis for judging the 
accuracy of the reconstructed model shape. In addition to 
this index, the evaluation of the digital reconstruction model 
of cultural heritage should also be done in terms of speed, 
cost, the ability to acquire complex forms, and the ability to 
acquire color information. 


User actions and model generation validation 
under two scenarios 

In the second phase of the experiment, two simple acquisi- 
tion environments for the multi-view reconstruction method 
were designed for inexperienced visitors, and the main goal 
was to avoid the generation of low-quality acquisition images 
while reducing instructions. 

The first scanning space solution consisted of two soft 
lights, an automatic turntable, and a pre-positioned camera, 
with the rotation of the turntable set to 18 degrees at a time 
and the vertical height of the camera set to three positions so 
that the line from the center of the turntable to the camera 
was angled at 20, 40, and 60 degrees from the plane of the ta- 
ble. The object is placed on the turntable, which rotates every 


time the camera shutter is pressed, and the camera position 
is adjusted upon completion of the turnaround, resulting in 60 
images for model generation. Anyone can use the space, and 
the user can easily get the information they need by pressing 
the camera's shutter button and changing the height of the 
camera when the turntable is done turning. 

The other space consists of only a light box and a manu- 
al turntable. The subject is placed on the turntable, and the 
user manually controls the turntable and uses phone to take 
multi-angle shots. Unlike the previous method, the user's cell 
phone is connected in real time to the computer responsible 
for model generation in order to visualize which images have 
been aligned and point cloud data generated, and which loca- 
tions need to be supplemented with new images. 


Figure 2. Two acquisition environments 


After a brief evaluation of the experience of the two process- 
es, the quality of the newly produced digital model is verified. 
Prior to this part of the validation, a high-precision laser 3D 
data was prepared and compared with the experimentally 
produced digital model to verify the accuracy of the latter. 


Results 


Model quality 

The quality of the final model was excellent even though there 
were some limitations in the real museum environment, 
such as the glass covering the collection, uneven lighting, 
etc., which created some obstacles to the image acquisition 
of the visitors. The digital models created from the images 
captured by the volunteers were not only true to form, but 
also contained color information with a lot of detail that can 
be used directly in digital scenes such as digital archives, film 
and television productions, and online exhibitions. 

In the first phase of the experiment, two visitors used a 
camera and a smartphone as the image acquisition devices to 
reconstruct the ” Bronze square ding (cauldron) with human 
faces” in 3D to obtain results A and B respectively (Figure 3). 
After aligning the two, the grid distance between A and B was 
calculated using A as a reference, and the average value of the 
mesh distance was about 0.0044, with no obvious error in the 
shape of the two (Figure 4). The accuracy of the digital shape is 
still guaranteed even when using a smartphone, which is inferi- 
or to acamera in terms of pixels and professional parameters, 
as the acquisition device. Although not as good as professional 
cameras in terms of color and grid detail, the more popular mo- 
bile devices have greater ability and potential to provide a large 
and useful digital asset of cultural heritage. 


Figure 3. Model results obtained by using a camera and a phone 
as acquisition devices in the museum. 


Figure 4. The mesh distance between the model obtained from the phone 
and the camera in the museum. 


Figure 5. Model results obtained by phone in the laboratory. 


Figure 6. The mesh distance between the model obtained from the phone 
and the laser scanner in the laboratory. 
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In a more well-lit and less constrained acquisition environ- 
ment, with real-time feedback of acquisition information, 
smartphones can perform 3D reconstruction faster and bet- 
ter, as reflected by a more detailed and color-accurate model 
mesh (Figure 5). In addition, if redundant image information 
is provided using different devices, the model quality can be 
further improved, although the computation time increases. 


User experience 

Most participants were pleased that they were able to cre- 
ate a digital resource about their cultural heritage and were 
willing to continue to participate in this work. Most visitors 
were more comfortable using their mobile devices to recon- 
struct objects in 3D than to scan them with a simple press of 
a shutter, although they may be tired of the process or unsure 
whether their images would produce valid results. In addition, 
the real-time feedback of scanning results on the screen or 
the use of AR to guide the user to the best shooting position 
can calm anxiety and enrich the experience of participating 
in the digital preservation of cultural heritage. 
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Conclusions 

In summary, visitors can use their own mobile devices to 
digitize cultural objects, although there are limitations in the 
current museum environment that affect the quality of the 
3D reconstruction results. At the same time, museum in- 
stitutions can value the ability and willingness of visitors to 
create cultural heritage image resources, actively provide 
opportunities for visitors to participate in digitizing cultural 
heritage, optimize the user experience of creating digital as- 
sets through AR and cloud computing, and provide new, fast, 
and sustainable paths for cultural heritage preservation and 
digitization. 

In the future, the role of visitors in museums will also grad- 
ually transform from being educated and entertained to being 
conservators and builders of cultural heritage. Public participa- 
tion in cultural heritage conservation is the key to addressing 
the current crisis in cultural field, and new and proven technol- 
ogies and products need to be brought to the public in order to 
awaken more potential forces and tap into hidden resources 
for this cause. 
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